Skip to content

Make benchmarks for getall more stable - #1189

Merged
asvetlov merged 2 commits into
aio-libs:masterfrom
asvetlov:stable-perf
Jun 22, 2025
Merged

Make benchmarks for getall more stable#1189
asvetlov merged 2 commits into
aio-libs:masterfrom
asvetlov:stable-perf

Conversation

@asvetlov

@asvetlov asvetlov commented Jun 22, 2025

Copy link
Copy Markdown
Member

Standard deviation for some benchmarks is too high, to reduce it the PR runs the test 30 times.

@asvetlov asvetlov added the bot:chronographer:skip This PR does not need to include a change note label Jun 22, 2025
@webknjaz
webknjaz requested a review from bdraco June 22, 2025 15:46
@codspeed-hq

codspeed-hq Bot commented Jun 22, 2025 •
edited
Loading

Copy link
Copy Markdown

CodSpeed Performance Report

Merging #1189 will degrade performances by 95.79%

Comparing asvetlov:stable-perf (22fe31e) with master (8303397)

Summary

❌ 12 (👁 12) regressions
✅ 232 untouched benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
👁 test_cimultidict_getall_istr_hit[c-extension-module] 23 µs 107.1 µs -78.52%
👁 test_cimultidict_getall_istr_hit[pure-python-module] 97.3 µs 1,783.6 µs -94.54%
👁 test_cimultidict_getall_istr_miss[c-extension-module] 14.9 µs 31.8 µs -53.22%
👁 test_cimultidict_getall_istr_miss[pure-python-module] 34.7 µs 309.9 µs -88.79%
👁 test_multidict_getall_str_hit[case-insensitive-c-extension-module] 25.5 µs 117.9 µs -78.34%
👁 test_multidict_getall_str_hit[case-insensitive-pure-python-module] 96.2 µs 1,785.2 µs -94.61%
👁 test_multidict_getall_str_hit[case-sensitive-c-extension-module] 23.3 µs 103.7 µs -77.53%
👁 test_multidict_getall_str_hit[case-sensitive-pure-python-module] 92.3 µs 2,190.9 µs -95.79%
👁 test_multidict_getall_str_miss[case-insensitive-c-extension-module] 17.6 µs 42.6 µs -58.66%
👁 test_multidict_getall_str_miss[case-insensitive-pure-python-module] 33.6 µs 307.4 µs -89.07%
👁 test_multidict_getall_str_miss[case-sensitive-c-extension-module] 15.4 µs 28.8 µs -46.69%
👁 test_multidict_getall_str_miss[case-sensitive-pure-python-module] 29.5 µs 529.4 µs -94.43%

@codecov

codecov Bot commented Jun 22, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.31%. Comparing base (8303397) to head (22fe31e).
⚠️ Report is 53 commits behind head on master.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master    #1189   +/-   ##
=======================================
  Coverage   98.31%   98.31%           
=======================================
  Files          27       27           
  Lines        3852     3856    +4     
  Branches      700      704    +4     
=======================================
+ Hits         3787     3791    +4     
  Misses         18       18           
  Partials       47       47
Flag Coverage Δ
CI-GHA 98.31% <100.00%> (+<0.01%) ⬆️
MyPy 80.38% <100.00%> (+0.02%) ⬆️
pytest 99.85% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@asvetlov
asvetlov merged commit 7e68e1b into aio-libs:master Jun 22, 2025
64 checks passed
@asvetlov
asvetlov deleted the stable-perf branch June 22, 2025 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bdraco bdraco Awaiting requested review from bdraco

Assignees

No one assigned

Labels

bot:chronographer:skip This PR does not need to include a change note

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@asvetlov